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Faculty perceptions of the scholarship of teaching and learning: 
Definition, activity level and merit considerations at one university 
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Abstract: This study investigated the status of the Scholarship of Teaching and 
Learning (SoTL), the amount and types of pedagogical scholarship, and the merit 
accorded SoTL within academic units for purposes of faculty assessment (i.e. 
hiring, promotion, and tenure decisions) at a research/doctoral granting 
institution. Responding to an electronic survey were 159 tenured and tenure track 
faculty (30% assistant professor, 31% associate, 31% full professors; 6% 
collateral/clinical/administrative) from cdl colleges and schools. The data 
analyzed showed a disconnect between the perceptions of tenured and non- 
tenured faculty on what qualifies as SoTL and the place of scholarly works within 
promotion and tenure. More specially, non-tenured faculty tended to judge 
teaching activities as SoTL regardless of their likelihood for publication, whereas 
tenured faculty more often recognized only published works as SoTL. Results cdso 
indicated that a limited number of faculty were engaged in SoTL; possible 
reasons could include lack of external funding and nebulous promotion and 
tenure guidelines as they concern SoTL. 

Keywords: educational research, scholarship of teaching and learning, promotion 
and tenure, faculty attitudes, faculty perceptions, college faculty 

I. Introduction. 

Investigating the processes and products of teaching in higher education has emerged as a 
noteworthy activity across all disciplines in the academy. Increasingly referred to as the 
scholarship of teaching and learning (SoTL), the importance of this category of scholarship, as 
well as what counts as the scholarship of teaching and learning, remains unclear and debatable in 
many universities and colleges. While there are a growing number of SoTL articles in both 
general education journals and journals specific to disciplinary pedagogy (Bernstein & Bass, 
2005; Brew, 2010; Burke, Johnson, & Kemp, 2010; Gurung, Weidert, & Jeske, 2010; Hubball, 
Clark, & Poole, 2010; Jurkowski & Kerr, 2010; Osborne, et. al., 2009; Persellin & Goodrick, 
2010; Sharmaa & McShaneb, 2008), the value of these activities, particularly within the 
evaluation process for full-time faculty, is often contested. The study reported in this article was 
undertaken to further develop this literature by investigating faculty perceptions about the types 
of scholarly or teaching activities that could be categorized as SoTL, the amounts and type of 
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pedagogical scholarship conducted by faculty at a research/doctoral granting institution and the 
merit accorded SoTL within different academic units of this institution for purposes of faculty 
assessment (i.e., hiring, promotion, and tenure decisions). The significance of this type of 
scholarship on the enhancement of teaching practices and pedagogy is discussed in the 
conclusion. 

II. Review of the Current Literature. 

Traditionally, scholarship has been demonstrated through a peer-reviewed, publicly disseminated 
product (Fincher & Work, 2006). However, in the early 1990s Boyer (1990) suggested a 
recognition of scholarship that went beyond discipline specific research to include research that 
advances knowledge about teaching. And, in an attempt to bring SoTL further in alignment with 
traditional disciplinary scholarship, the Journal of the Scholarship of Teaching and Learning (as 
cited by Shulman (1999) suggested that knowledge about teaching and learning be considered 
scholarship if it manifested the following three characteristics: “It should be public , susceptible 
to critical review and evaluation, and accessible for exchange and use by other members of one's 
scholarly community” (p. 5). 

However, a lack of consensus about what is meant by SoTL persists. McKinney (2006) 
offers two areas of contention as reasons for the lack of clarity: the confusion between what is 
scholarly and what is scholarship, and the debate about the overlap or relationship between SoTL 
and the assessment of student achievement within a course. McKinney suggests that these two 
areas of contention are barriers to the advancement of SoTL within academia. SoTL, she 
suggests, is scholarship that overlaps with the more traditional discipline specific scholarship of 
discovery, integration, and application and further argues that SoTL should be considered to be 
as important as disciplinary specific research. We integrated McKinney’s (2004) approach to 
SoTL with the characteristics set forth above by the Journal of Scholarship of Teaching and 
Learning and articulated a working definition of SoTL-“the systematic, literature-based study of 
processes and outcomes involved in teaching and learning intended for peer-reviewed 
publication and dissemination”-for use as a beginning frame of reference for this study. 

How has scholarship of teaching and learning been explored in other studies? Gurung 
Gurung, Ansburg, Alexander, Lawrence, and Johnson (2008) surveyed members of North 
American psychology departments regarding SoTL. Participants were asked to rank SoTL 
products by the product’s importance in personnel decision making. Of the 142 participants, over 
half of the participants endorsed peer-reviewed publications (54.9%) and leading faculty 
workshops (55.6%) as important to personnel decision making. Over one-third of participants 
identified attending faculty workshops (44.3%). Professional presentations (40.0%), evidence of 
teaching impact (45.3%), receipt of a grant (35.5%), and development of a portfolio (32.6%) 
were also identified as being important. Clearly, not all of these products are comparable to more 
traditional disciplinary scholarship products, heightening the confusion about what is SoTL and 
what is not. 

A number of articles suggest that while SoTL is acceptable within higher education, it 
has not yet been acknowledged as equal to discipline specific research (McKinney, 2006; 
Walker, Baepler, & Cohen, 2008). Gurung, et. al. (2008) posit that although formal policies 
regarding SoTL in psychology departments were reported by participants, it seemed that these 
policies had not yet been fully institutionalized; departmental policies encouraged SoTL 
activities while institutional policies did not include such encouragement. Participants’ 
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perceptions of support for SoTL were higher at the department level than at the institutional 
level. Shapiro (2006) asserts that SoTL activities are considered “add-ons” (p. 42) to the 
expectations that faculty will get competitive funding, publish refereed articles, and develop 
national reputations all within their specific discipline. He states that SoTL contributions do not 
change or replace the traditional productivity demands. 

While the argument has been made that SoTL should be valued and rewarded in the same 
way as traditional scholarship (Fincher & Work, 2006), many proponents of SoTL are concerned 
that such productivity is not as positively recognized nor as highly rewarded. McKinney (2006) 
posits that older faculty members have a narrower view of scholarship and hold stereotypical 
views of SoTL work. Included in these stereotypes are the beliefs that SoTL work is of lower 
quality, cannot be generalized, and involves questionable peer review processes. Gurung, et. al. 
(2006) state that SoTL activities were not perceived as being relevant criteria in the reward 
structure that includes merit pay and hiring decisions. Although some participants in this study 
did believe that SoTL had some influence in tenure and promotion decisions, more of them 
reported that SoTL was not mentioned in evaluation guidelines. 

It has been suggested that although the traditional reward structure includes teaching, 
research, and service equally, good teaching is actually expected but not privileged (Walker, 
Baepler, & Cohen, 2008). Supporting this position, Terpstra and Honoree (2009) suggest that the 
emphasis of a university can be assessed by the nature of its reward structure. Their study 
indicated that the most effective teachers, those with better ratings of student knowledge and 
competency and student satisfaction, worked in systems that emphasized research and teaching 
equally. These faculty members were also significantly more satisfied in terms of both their jobs 
and compensation. University emphasis on research produced more productive faculty, measured 
in both research quality and quantity, but these faculty members had lower satisfaction ratings as 
well as lower effective teaching ratings. These findings suggest that though teaching may be 
identified as an important activity, in actuality, the reward structure is still heavily based on 
research. 

In order to help define SoTL and advance it as legitimate scholarship toward promotion 
and tenure in our own institution, members of a multidisciplinary faculty learning community 
(FLC) agreed to pursue a project which would provide some insight about SoTL derived from 
the faculty at our large urban university. The FLC developed a survey and disseminated it 
university-wide to investigate 1) the activities and products that faculty considered to be SoTL, 
2) the amount and types of pedagogical scholarship, and 3) the merit accorded SoTL within 
academic units for purposes of faculty assessment (i.e. hiring, promotion, and tenure decisions) 
within their school or college. We report here the outcomes of this survey and offer implications 
as to the impact of SoTL on higher education. 

III. Methodology. 

A. Participants and Data Collection Procedures. 

A list of all tenured (n= 737), tenure-track (n=274), and collateral/clinical faculty (n= 913) of this 
one large Carnegie-designated research/doctoral granting institution in the southeastern United 
States was obtained from the institution’s human resource department. Those on the list received 
an e-mail request to complete this IRB-approved electronic survey via a secure Internet server. 
No incentives were given to complete the survey. The survey remained open for three weeks. 
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Faculty members who did not complete the survey were sent reminders after the first and second 
weeks. This scheme and time frames were described by Bailey (1991). 

Seventeen percent (17%) of 274 tenure track faculty, 13% of 737 tenured faculty, and 2% 
of 913 collateral/clinical faculty responded to the survey to yield a convenience sample of 159 
study participants. Although low, our overall response rate for tenure and tenure-track faculty 
(15%) was within the range of other university faculty surveys, such as 22% for a National 
Faculty Survey, APLU Sloan National Commission on Online Learning (Seaman, 2009), and 
15.8% (Maurer, et. al., 2009). Further limitations of the study are addressed in the Conclusion 
section of this article. 

B. Instrument. 

As noted above, we articulated a definition of SoTL for use as a frame of reference from which 
to generate lists of scholarly activities and vignettes and other questions for the survey. We 
included this same definition "the systematic, literature-based study of processes and outcomes 
involved in teaching and learning intended for peer-reviewed publication and dissemination" at 
the beginning of the survey to provide a common definition of SoTL for the respondents. The 
survey was piloted on faculty members recently retired from the university where the survey was 
administered. Revisions were made based on the suggestions from the survey pilot. The final 
survey questions generated from our FLC’s overall research questions were: (1) What activities 
and scholarly products are considered as SoTL? (2) What is the amount and type of pedagogical 
scholarship generated by faculty in the institution? (3) What merit is accorded SoTL for purposes 
of faculty assessment? 

In order to answer the first of our FLC’s research questions about the types of activities 
and scholarly products considered as scholarship of teaching and learning at the university, we 
developed a list of 16 scholarship related activities that could be interpreted as SoTL according 
to the McKinney definition (McKinney, 2006). Respondents were asked: “From the following 
list, indicate the extent to which you would consider each to be part of the Scholarship of 
Teaching and Learning (SoTL)?” (1 = Definitely a part of SoTL; 2 = Probably a part of SoTL; 3 
= Probably NOT a part of SoTL; 4 = Definitely NOT a part of SoTL). Examples of the types of 
activities were: Data-driven, classroom-based studies; Reviews of the pedagogic literature; 
Authoring textbook or textbook chapters; and Developing a new course (see Appendix I for 
complete list of activities). Subsequently at the data analysis stage, the value categories were 
collapsed into definitely or probably a part of SoTL (1) and probably not or definitely not (3) in 
order to maximize variation between activities considered or not considered as SoTL for 
subsequent data analysis. 

The second research question concerning the amount and types of SoTL across the 
various disciplines at the university was answered by asking respondents three questions: 1) How 
many publications (published or accepted for publication) they authored that met the given 
definition of SoTL; 2) The nature of their SoTL publications (1 = Mostly empirical; 2 = About 
evenly divided between empirical and conceptual; 3 = Mostly conceptual; 4 = Other); and 3) 
Whether they had ever received external funding for SoTL (1 = Yes or 2 = No). The number of 
publications was subsequently collapsed into none = 0, 1 to 5 = 1, and more than 5 = 2 based on 
the distribution of the responses generated by the respondents. 

For the third research question about the merit accorded the scholarship of teaching and 
learning for promotion and tenure and other award decisions, we developed five vignettes 


Journal of the Scholarship of Teaching and Learning. Vol. 11, No. 3, August 2011. 
www.iupui.edu/~josotl 


4 



Secret, M., Leisey, M., Lanning, S., Polich, S. and Schaub, J. 


describing different types of intellectual property: 1) Publication of a data-driven classroom- 
based study in a peer-reviewed journal; 2) Publication of an article, conceptual in nature, about 
academic leadership in a peer-reviewed journal; 3) Presentation of a data-driven classroom-based 
study in at a prestigious national conference; 4) A series of case studies published in a peer- 
reviewed online database; and 5) Faculty to Student Social Media. Following from the 
McKinney definition of SoTL noted above, these vignettes represented the five most commonly 
noted events recognized for promotion and tenure at this institution (See Appendix 2 for 
descriptions of the full vignettes). Respondents were asked to indicate if they thought each of the 
intellectual properties should be used for making decisions about 1) Promotion and tenure; 2) 
Annual review for merit-based pay raises; and 3) Award consideration, including academic unit, 
university, or external awards. Response categories for each of these questions were: 1 = 
Definitely consider; 2 = Probably consider; 3 = Probably not consider; 4 = Definitely not 
consider. 

We developed scales for overall merit consideration for each of the five intellectual 
property vignettes. A scale for each of the intellectual property items was developed by summing 
the responses of the three decision items (promotion and tenure, annual merit, and award) for 
each of intellectual property vignettes. Scaled scores ranged from 3 to 12 for each vignette, with 
lower scores indicating more overall merit for that particular intellectual property vignette. 
Univariate statistics for each the scales are shown in Table 1. The intellectual property item 
“Publication of a data-driven classroom-based study in a peer-reviewed journal” was considered 
to have the most overall merit (m = 4.13) for faculty promotion and tenure, raises, and awards, 
while social media had the least merit (m = 8.81). 


Table 1. Merit Consideration for Intellectual Property Vignettes. 


Intellectual Property Item 

Mean 

Median 

SD 





Publication of a data-driven classroom-based study in 
a peer-reviewed journal: 

4.13 

3.00 

1.59 

Publication of an article, conceptual in nature, about 
academic leadership in a peer-reviewed journal: 

5.00 

4.00 

2.05 

Presentation of a data-driven classroom-based study in 
at a prestigious national conference: 

4.65 

4.00 

2.04 

A series of case studies published in a peer-reviewed 
online database: 

4.88 

4.00 

2.09 

Faculty to Student Social Media: 

8.81 

9.00 

2.62 


We also asked if respondents thought that the specific intellectual property belonged in 
either or all of the sections of faculty dossier or vitae - teaching, research, or service. Response 
categories for each of these sections were: 1 = Definitely belongs; 2 = Probably belongs; 3 = 
Probably does not belong; 4 = Definitely does not belong. Respondents also provided 


Journal of the Scholarship of Teaching and Learning, Vol. 11, No. 3, August 2011. 
www.iupui.edu/~josotl 


5 




Secret, M., Leisey, M., Lanning, S., Polich, S. and Schaub, J. 


information about their university rank (assistant, associate, full professor, other), department or 
unit affiliation 6 , and number of years of employment as a full-time faculty member at this 
particular university. However, in order to protect the anonymity of the respondents, additional 
identifying information such as sex, race, ethnicity or age was not asked. 

C. Data Analysis. 

Univariate statistics were used for pre-analysis data screening to check for missing data, outliers 
and keying errors and to explore the distribution and characteristics of the variables (Mertler & 
Venetta, 2002). Not surprisingly, data from this non-probability sample did not meet the 
assumptions necessary for parametric tests; thus non-parametric statistical tests were chosen to 
analyze the data (Mertler & Vannatta, 2005; Moore, McCabe, & Craig, 2009). The Chi square 
test of independence, the most commonly used non-parametric test statistic (Vogt, 2007) was 
used to examine the relationship between the nominal variable of SoTL activities and the ordinal 
variables of faculty rank and discipline. Spearman’s rho, a non-parametric measure of rank 
orderings between two interval level variables, was used to examine the relationship between the 
interval level variables; Mann-Whitney U, a non-parametric test for a difference between two 
groups and Kruskal-Wallis, a non-parametric test for the difference between three or more 
groups, were used to examine relationships between nominal and interval level variables 
(Rosenthal, 2001). 

We grouped the respondents by colleges/schools; if there were fewer than five 
respondents from a particular school/college, we further grouped those respondents with 
associated colleges/schools in order to ensure anonymity of the respondents. For example, 
respondents from schools/colleges with fewer than five participants and who were associated 
with the medical campus were grouped as Allied Health, Dentistry, Pharmacy and Nursing; those 
associated with professional programs on the academic campus and with fewer than five 
respondents were grouped with Professional Schools; and, those associated with the 
undergraduate arts and sciences programs on the academic campus were grouped as Arts and 
University College. Our final school/college grouping was: Humanities and Science; Arts and 
University College; Allied Health, Dentistry, Pharmacy and Nursing; Professional Schools 
(Education, Business, Engineering, and Social Work); and School of Medicine. 

IV. Findings. 

A. Faculty Characteristics. 

Based on the college/school grouping noted above, the distribution of the 159 respondents was: 
Humanities and Science (n = 50, 33% of respondents); Allied Health, Dentistry, Pharmacy and 
Nursing (n = 23, 15% of respondents); Arts and General Education (n = 26, 17% of 
respondents); Professional Schools of Education, Business, Engineering, and Social Work (n = 
23, 17% of respondents); and the School of Medicine (n = 28, 18% of respondents). Respondents 
varied greatly in the number of years as full time faculty at the institution—from less than 1 to 
38, with 8 years being the median and a mean of 12 years (SD 10.31). Of the 159 faculty 


6 The administrative structure of this University is divided into a medical campus and an academic campus and further organized 
into 12 identifiable colleges/schools across both campuses. The majority of colleges/schools are further divided into various 
departments or organizational units with some departments having fewer than five faculty members while some have over 100. 
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members who responded to the survey, 49 (31%) identified as full professors; 50 (31%) 
identified at the associate level; 46 (29%) identified at the assistant professor level; and 14 (9 %) 
identified as collateral/clinical or administrative that we labeled “other.” Because the small 
number of faculty in the “other” rank yielded too small a cell size to be usable for the chi-square 
test analysis, only full, associate, and assistant level professors were included in the bivarate 
analysis for the faculty rank variable. 

B. Question 1: What Activities are Considered as Scholarship Of Teaching And Learning? 

We answered this question by examining differences of opinion about SoTL activities based on 
faculty rank, years as full-time faculty, and disciplinary division. 

Figure 1 shows the percentage of faculty, by faculty rank, who reported definitely or 
probably be considered as SoTL for each of the 16 activities. A great majority of respondents 
across all ranks considered dissemination of knowledge activities to be SoTL: Data-driven, 
classroom-based studies (98%) ; Publication of any in peer-reviewed journals (97%); Publication 
of any in peer-reviewed electronic databases (92%); Case studies of teaching and learning 
(89%); Presentation of any at regional, or national conferences (95%). On the other hand, the 
following activities, not generally associated with knowledge dissemination, were considered 
SoTL by fewer number of faculty: Making changes to your teaching based on those evaluations 
(23%); Educational blogs and wikis (38%); Using Blackboard in your teaching (17%); Putting 
your lecture notes on PowerPoint (12%). 

A Chi square test of independence was used to examine the relationship between 
activities considered as SoTL and faculty rank. Statistically significant differences by rank were 
found for the following five out of 16 activities: 

• Developing case studies for use in the classroom (x 2 = 14.38; p = 0.001) 

• Developing a new course (x 2 = 8.09; p = 0.017) 

• Integrating technology into your teaching (x 2 =6.71; p = 0.035) 

• Making changes to teaching based on student evaluations (x 2 = 12.77; p = 0.002) 

• Using Blackboard (x 2 = 6.65; p = 0.036) 

Interestingly, SoTL activities that were differentiated by rank were those not generally 
associated with publication or knowledge dissemination. For example, only 40% of tenured full 
professors believed that “Developing case studies for use in the classroom” was SoTL while 78% 
of tenure-track assistant professors thought so. Similarly, only 18% of tenured, full professors 
believed that “Making changes to teaching based on student evaluations” was SoTL while 48% 
of tenure-track assistant professors thought the same. 

Given the differences in what activities were considered to be SoTL based on faculty 
rankings, it was not surprising that there were differences based on years as a full time faculty at 
the university. Mann-Whitney U test of differences between two groups (1 = activity is 
considered SoTL; 2 = activity is considered not SoTL) revealed that respondents with fewer 
years of teaching were more likely to consider the following activities as SoTL: Developing case 
studies for use in teaching (Mann-Whitney U = 1715.500; p = 0.007); Putting lectures on 
PowerPoint (Mann-Whitney U = 724.500; p = 0.044); and Making changes to teaching based on 
student evaluations (Mann-Whitney U = 1188.500 ;p = 0.000). 
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Figure 1. Activities Considered as Scholarship of Teaching and Learning by Rank. 
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Opinions of faculty in the colleges/schools differed in only one of the 16 activities, 
“Integrating technology into your classroom” (x = 9.39 p = 0.05) with respondents from the 
Professional Schools division least likely to report this activity as SoTL. 

C. Question 2: What is the Amount and Type of Pedagogical Scholarship generated by faculty 

in this institution ? 

Faculty members’ response to the number of SoTL publications they had produced, the extent to 
which these publications were empirical or conceptual in nature, and whether or not they 
received external funding to support their SoTL publications follows. Note that we asked 
respondents to refer to the definition of SoTL provided in the survey instrument, that the 
designation of empirical or conceptual was left to the interpretation of the respondent, and that 
external funding varied by yes or no rather than by amount of funding. 

The distribution of the responses to the question about number of SoTL publications 
ranged from 0 to 130 and was severely skewed (m = 6.18, med = 1.00, mode = 00, sd = 19.43). 
Thus, we collapsed this variable into the following categories (0 = 0 publications; 1 = 1-5 
publications; and 2 = more than 6 publications. The majority of respondents (43%) reported 
having no SoTL activity, 39% reported 1-5 publications and 14% reported having more than 6 
publications. About a third (37%) of respondents reported that their SoTL publications were 
about evenly divided between being empirical and conceptual in nature, while 28% reported that 
their SoTL publications were primarily empirical, and 17% reported their publications being 
primarily conceptual in nature. Only 23% of respondents reported receiving external funding to 
support their SoTL work. 

Kruskal-Willis tests were used to analyze number of publications by college/unit and by 
faculty rank, and chi-square tests were used to examine the relationships between external 
funding, type of SoTL article published, unit/college, and faculty rank. None of these 
relationships reached statistical significance except for the relationship between external funding 
and faculty rank (x = 6.187 \p = 045). Associate and full professors were more likely to receive 
external funding for their SoTL efforts than were assistant professors. 

D. Question 3: What Merit is Accorded SoTL for Purposes of Faculty Assessment? 

We answered this question in two ways. First, we reported the percentage of faculty who 
considered each of the vignettes for promotion and tenure, annual raises and for award decisions 
(Figure 2) and the percentage of faculty who would place the various intellectual properties in 
the various sections of their vita (Figure 3). The majority of respondents reported that the 
following intellectual properties probably or definitely should be considered in the faculty 
assessment for annual review, for promotion and tenure, and for awards; and also that these same 
intellectual properties belong in the “Research” section of the dossier: 

a) Publication of a data-driven classroom-based study in a peer-reviewed journal: 87% 
annual review; 95% promotion and tenure; 99% awards 

b) Publication of an article, conceptual in nature, about academic leadership in a peer- 
reviewed journal: 82% annual review: 93% promotion and tenure; 95% awards 

c) Presentation of a data-driven classroom-based study in at a prestigious national 
conference: 78% annual review; 91% promotion and tenure; 93% awards 
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d) A series of case studies published in a peer-reviewed online database: 75% annual 

review; 90% promotion and tenure; 93% awards. 

Faculty to student social media was considered appropriate by the majority of faculty 
(66%) only for placement in the “Teaching Section” of the vita. The majority of respondents 
would not place any of the five activities in the “Service” section of the dossier. However, 26% 
would place the “Publication of an article, conceptual in nature, about academic leadership in a 
peer-reviewed journal” and 18% would place “A series of case studies published in a peer- 
reviewed online database” in the Service section of the vita. 

The second way we answered this question was to explore differences in the overall merit 
intellectual property scales (described in methodology section above) across schools/colleges and 
faculty rank. Kruskhal-Wallis test of difference between means indicated no statistically 
significant differences across schools/colleges. However, full time faculty considered the 
intellectual property item ‘faculty to student social media’ to have less overall merit (m = 9.74) 
than did assistant professors (m = 8.29) (K-W chi square = 8.55; p = 0.01). 

IV. Conclusion. 

A. Interpretation of Results. 

This investigation was performed to add to the rather limited empirical data on faculty 
perceptions of SoTL. Our study gathered information about faculty opinions of SoTL, the 
amount and types of scholarship it represented, and the value given to SoTL within the context 
of faculty evaluation. Critical to establishing faculty evaluation criteria is defining what 
constitutes SoTL and the merit accorded to it. Our faculty, regardless of rank, considered the 
following activities as SoTL: Data-driven classroom-based published study, educational 
administration publication, data-driven classroom-based national presentation, and case study 
series publication. These activities were also given merit consideration for promotion and tenure 
and placed within the research component of the dossier. Thus, these activities help define what 
our faculty considered SoTL. 

-A commonality of the activities considered as SoTL by a great majority of faculty across 
ranks is that they are generally reviewed, evaluated, and made accessible to others. We can say 
then, at least at this institution, the same characteristics that define traditional discipline-based 
research also apply to SoTL. However, five of the 16 teaching activities that are less likely to be 
publishable scholarly products (Developing case studies or problem sets for use in your 
classroom; Developing a new course; Integrating technology into your teaching; Making changes 
to your teaching based on those evaluations; Educational blogs and wikis) were considered to be 
SoTL by approximately half of the junior faculty whereas only a fraction of senior faculty 
considered these same activities as SoTL. This is an important disconnect found between the 
perceptions of tenured faculty and their non-tenured colleagues. It appears that our junior faculty 
do not see a key distinction between good, initiative, or scholarly teaching and SoTL as 
described by Glassick, Huber and Maeroff (1997) and supported by others (Cambridge, 1999; 
Hutchings & Shulman, 1999; Shulman, 1999). Glassick and colleagues promoted scholarly 
teaching as the investigation of teaching and learning by a variety of systematic analytical 
methods. Results are used to refine teaching methods and improve student learning. The 
scholarship of teaching and learning contains all of the elements of scholarly teaching but further 


Journal of the Scholarship of Teaching and Learning , Vol. 11, No. 3, August 2011. 
www.iupui.edu/~josotl 


10 


Secret, M., Leisey, M., Lanning, S., Polich, S. and Schaub, J. 


Data-driven Classroom- 
based Published Study 


Educational 

Administration 

Publication 


Data-driven Classroom- 
based National 
Presentation 


Case Study Series 
Publication 


Faculty to Student Social 
Media 



□ Definitely/Probably Annual Review 

■ Definitely/Probably Promotion/Tenure 

□ Definitely/Probably Award Consideraton 


10 


20 


30 


40 


50 


60 


70 


80 


90 


100 


Percent of Respondents 


Figure 2. Intellectual Property Merit Considerations. 


Journal of the Scholarship of Teaching and Learning, Vol. 11, No. 3, August 2011. 
www.iupui.edu/~josotl 


11 












































































Secret, M., Leisey, M., Lanning, S., Polich, S. and Schaub, J. 


Data-driven Classroom- 
based Published Study 


Educational 

Administration Publication 


Data-driven Classroom- 
based National 
Presentation 


Case Study Series 
Publication 


Faculty to Student Social 
Media 



□ Teaching Definitely/Probably Belongs 
■ Research Definitely/Probably Belongs 

□ Service Definitely/Probably Belongs 


0 10 


20 30 40 50 60 70 80 


90 100 


Percent 


Figure 3. Placement of Intellectual Property in Vita. 


Journal of the Scholarship of Teaching and Learning, Vol. 11, No. 3, August 2011. 
www.iupui.edu/~josotl 




































































Secret, M., Leisey, M., Lanning, S., Polich, S. and Schaub, J. 


extends the process by making public the results of the investigation. Our faculty may benefit 
from having these similarities and distinctions clarified. 

Our findings suggest that socialization as described by McKinney (2006) is not the 
reason for varying perceptions of SoTL activities among tenured and non-tenured faculty. She 
offered that socialization of new and junior faculty results in scholarship being understood only 
as traditional disciplinary research; this sends the message that SoTL is a component of teaching 
and is less important than discovery scholarship in the discipline. It appears that the majority of 
our faculty do not discount the merit of SoTL as compared to disciplinary-based research, as 
long as the scholarly product is critically reviewed and made available to others. 

Nearly half of faculty (43%) responding to our survey reported no SoTL experience. 
Even though not completely engaged in SoTL, this faculty population judged sample 
productivity activities resulting in dissemination of information through peer reviewed journals 
or online databases appropriate for annual review and/or promotion and tenure consideration. 
This is consistent with our thoughts above on what is traditionally thought of as scholarship 
(Fincher & Work, 2006). Gurung, et. al. (2008) reported that the majority of faculty respondents 
also agreed that any peer reviewed work was worthy of consideration for promotion and tenure. 
For example, his psychology faculty and ours indicated presentation at professional conference 
and reflective essay on teaching as SoTF. 

Our study investigated faculty experience by the number and types of SoTF publications, 
(either empirical or conceptual in nature), amount of external funding, and differences with 
SoTF activity among divisions. Anecdotal evidence at our university suggests that there are 
varying interpretations of promotion and tenure guidelines among colleges/schools where the 
amount and type of SoTF publication and obtainment of external funding may be the criteria 
used to judge whether activities are worthy in the faculty assessment process. We found 
differences between colleges/schools. Humanities had the least number of publications and 
Professional schools the most. Professional schools also had the greatest amount of external 
funding. Healey (2005) states, in regards to divisional or disciplinary approaches towards SoTF, 
that the “.. .conduct of research and teaching methods tend to differ between disciplines” because 
SoTF involves critical reflection of one’s own practice, development of subject-based teaching 
modalities, and discipline specific pedagogic research (Healey, 2000). Thus, our differences 
among colleges/schools in terms of the number of publications and amount of external funding 
may reflect varying disciplinary cultures or norms about what is considered valuable and what 
qualifies as research or scholarship. However, there was only a statistical difference for one of 16 
sample SoTF activities among faculty by colleges/schools, which could have been by chance. 
Our results suggest that faculty perceptions of what constitutes SoTF were similar across 
colleges/schools, yet faculty within Professional schools appear to be more actively engaged in 
SoTF as measured by number of publications and external funding. 

Overall, only a quarter of surveyed faculty who indicated that they engaged in SoTF 
reported receiving external funding. Gurung, et. al. (2008) reported that almost 75% of faculty 
respondents indicated that neither departments nor institutions were providing adequate financial 
support for SoTF activities. Sixty-two percent of faculty respondents in the study by Pienta 
(2004) agreed that lack of financial support was a reason for varying rates of publications by 
chemical education faculty. Funding for academic pursuits offers financial resources for 
completion of proposed projects and suggests merit as projects are usually peer reviewed and 
awarded based on their value and potential for new discovery. Funding is generally 
acknowledged and awarded during faculty assessment. Yet some points for discussion naturally 
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follow: Does lack of funding preclude the merit of SoTL? It is our impression that there are 
fewer prestigious and financially rewarding opportunities for SoTL endeavors compared to 
traditional disciplinary-based research. Would SoTL be looked upon differently if it were highly 
funded? Is any scholarly pursuit that is not funded highly valued by contemporary institutions? 
Is this the proverbial “chicken or egg” scenario? That is, are there fewer fundable opportunities 
because SoTL, in general, is not highly valued, or is SoTL not highly valued because there are 
fewer fundable opportunities? 

The majority of our faculty respondents considered the same examples of SoTL activities 
for both “research” and “teaching” sections of a promotion and tenure dossier. This again 
suggests that interpretation of promotion and tenure guidelines at our university are either ill 
defined, misunderstood, or at the least, open to interpretation. It stands to reason that if our 
respondents were confused about where SoTL activities best fit, so could department chairs and 
promotion and tenure committee members. Certainly, there is an historical and negative notion 
of “double dipping” that is of concern, but we must ask: cannot a work hold more than one facet? 
For example, developing a new instructional program may fall under the “teaching section,” 
while publishing a peer-reviewed journal article featuring rigorous analysis of the program’s 
qualitative outcomes may be part of the “research section.” Terpstra and Honoree (2009) suggest 
that university outcomes are greater when there is a dual focus on teaching and research. Could it 
be that academic pursuits such as SoTL are highly complicated and integrated and naturally fall 
into more than one traditional promotion and tenure section? At the very least, our findings 
indicate that the faculty members compiling tenure and/or promotion materials should 
communicate in advance with tenure and promotion decision-makers about the place of said 
materials in a portfolio. At the same time, faculty would be best served to maintain written 
reflections on how their teaching informs their research, and vice versa. 

B. Limitations. 

There were inherent limitations of our study’s design. Although low, our overall response rate of 
15% was within the range of other university faculty surveys, such as 22% for National Faculty 
Survey, APLU Sloan National Commission on Online Learning, (Seaman, 2009) and 15.8% 
(Maurer, et. al., 2009). A particularly small percentage of collateral and clinical faculty 
responded to our survey. We speculate that this is due to a lack of motivation by these faculty to 
respond to a survey on promotion and tenure criteria as this is not relevant to their affiliation 
with the university. The concern with lower response rates in any investigation is that the 
findings are not universal but based on the attitudes of a few. It stands to reason that potential 
respondents with a vested interest in a particular topic are the most likely to respond. However, 
nearly half of our respondents indicated no SoTL experience. It was also interesting that almost 
70% of respondents were non-tenured faculty, which suggests that our study’s results more 
heavily reflect this group’s opinion than tenured faculty members’ opinion. Finally, pooling of 
faculty data from various colleges/schools may diminish details or nuances from the specialty 
areas but was done to protect faculty anonymity and provide a global view of faculty opinion. 

C. Implications. 

This investigation provides empirical evidence regarding faculty perceptions of the status of 
SoTL at one large urban university where the workload of many faculty centers on teaching. Our 
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findings further define SoTL and may serve as an impetus for revising promotion and tenure 
policies and procedures. Our results suggest that the majority of faculty agreed that scholarly 
works such as educational administration and case study series publications constituted SoTL. 
Discouragingly, though, our data indicated a disconnect between tenured and non-tenured faculty 
where non-tenured faculty more often judged teaching activates that are less likely to be 
published as SoTL. Our results also show that the majority of faculty considered the same 
examples of SoTL activities for both “research” and “teaching” sections of a promotion and 
tenure dossier. With the type and magnitude of variation found with our faculty’s interpretation 
of promotion and tenure guidelines, it is logical to call for the revision of these criteria so that 
faculty have consistent and equal opportunity for advancement. In accordance with many of our 
tenured faculty opinion and other authors (Cambridge, 1999; Glassick, Huber, & Maeroff, 1997; 
Hutchings & Shulman, 1999; Shulman, 1999) scholarship may only be recognized once a work 
is critically reviewed and disseminated to others. Guidelines may also offer acceptable venues 
for making scholarly work public. This could include electronic or paper journals and databases 
and presentation at a national conference where submissions have been peer reviewed prior to 
acceptance. Even so, presentation of course outcomes at department meetings or self reflection 
on one’s teaching practices that remains relatively private may be considered scholarly teaching 
and included under the “teaching” section of promotion and tenure dossier. Revising evaluation 
guidelines and making distinctions between what constitutes “teaching” and “research” could 
give faculty clearer direction and motivation. That is, if teaching faculty know they need to 
publish in order for their work to be recognized as “research” they may be more motivated 
towards developing systematic analyses for validating their teaching outcomes and making their 
work public. 

There are many potential benefits to making one’s scholarly work public within higher 
education. The faculty member engaging in SoTL may enhance his or her own teaching through 
critical reflection and external review. Raising awareness of an area of study within the academy 
can stimulate collaborative relationships and growth by bringing people together with like 
interests and expertise. Making scholarly work public through written assays and oral 
presentations, for example, can bring notoriety to university programs. Finally, sharing of 
teaching materials and determining, through systematic analysis, what teaching methodologies 
are most effective can decrease faculty workload, make curricula more efficient, advance 
pedagogy, and enhance student learning. 

It must be said that in order for faculty at this institution and others to fully engage in 
SoTL, policy and procedures may need to be modified. At the institutional level, for example, 
policies and procedures regarding sabbatical leave eligibility, faculty release time, internal grant 
funding and Institutional Review Boards’ approvals may need revision to support faculty 
engaged in this type of scholarship (“Policies and Procedures,” retrieved 2011). External 
accrediting agencies and disciplinary societies can also foster faculty work in this area by writing 
policies acknowledging and rewarding SoTL and the advancement of pedagogical research 
(“Policies and Procedures,” retrieved 2011). 

Time will tell what impact this investigation has locally and beyond. A logical first step, 
however, is to revise evaluation criteria to decrease the chance for varied interpretation across all 
faculty ranks. The point that seems to need the most clarification centers on what constitutes 
“research.” If faculty understand that “research” is a public event requiring rigorous review, they 
may be more motivated to design methodical approaches to analyzing their teaching 
effectiveness and student learning. From there, as described above, there are many potential 
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benefits within higher education. The focus of future research recognizing SoTL may include 
increases in the quantity and quantity of SoTL publications explicitly documenting advancement 
of student learning; faculty collaboration within and across disciplines at the local, regional and 
national level; and educational research with internal and external grant support. Finally, we 
propose that knowing the attitudes of one faculty toward SoTL might initiate momentum towards 
institutional reform here and elsewhere, providing equal opportunities for faculty promotion and 
evidence of pedagogical advancement. 

Appendix 1. Faculty Related Activities for Consideration in Promotion and/or Tenure. 

(List of scholarship related activities that could be interpreted as SoTL according to the 
McKinney definition, used in the faculty survey; from section Illb): 

a) Data-driven, classroom-based studies: i.e., formal research projects with appropriate 
statistical analysis, formal hypotheses and their testing, etc., employing either quantitative or 
qualitative methodologies 

b) Reflective essays on teaching: integrative evaluations of other work, essays that challenge 
current teaching and learning principles and practices, as well as encourage experimentation 
in the classroom 

c) Reviews of the pedagogic literature 

d) Case studies of teaching and learning 

e) Developing case studies or problem sets 

f) Publication of any of the above (a through e) in peer-reviewed journals 

g) Publication of any of the above (a thorough e) in peer-reviewed electronic databases 

h) Formal presentation of any of the above (a through e) to peers within your academic unit or 
institution 

i) Presentation of any of the above (a through e) at disciplinary, multi-disciplinary, regional, or 
national conferences 

j) Developing a new course 

k) Integrating technology into your teaching 

l) Reflecting on course evaluations and making changes to your teaching based on those 
evaluations 

m) Authoring a textbook or textbook chapters 

n) Putting your lecture notes on Powerpoint 

o) Using Blackboard 
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Appendix 2. Description of Vignettes. 


Vignette Title 

Description of Vignette as it appeared in survey 



p) Publication of a data- 
driven classroom-based 
study in a peer-reviewed 
journal 

An article published in Advances in Physiology Education 
describes the results of a study of undergraduate students in 12 
courses at 8 different institutions. The students were surveyed to 
determine the prevalence of 13 different misconceptions 
(conceptual difficulties) about cardiovascular function. Advances 
in Physiology Education is an online, peer-reviewed journal 
published by the American Physiological Society. 

q) Publication of an article, 
conceptual in nature, 
about academic 

leadership in a peer- 
reviewed journal 

A paper published in Academic Leadership provides historical 
perspective, definition, and implications for intellectual property, 
copyright, and fair use in education legislation. Academic 
Leadership is an on-line peer-reviewed journal that published 
articles by faculty and administrators from all disciplines and units 
in the academy. 

r) Presentation of a data- 
driven classroom-based 
study in at a prestigious 
national conference 

A study compared two general chemistry courses taught in the 
same semester by the same instructor. One section was taught 
using standard lecture format, while the second section was taught 
by substituting one lecture with a break-out session involving peer 
led group work. Student progress in the two sections was 
compared based on test results. The results of the study were 
presented at the most prestigious nation chemistry conference. 

s) A series of case studies 
published in a peer- 
reviewed online 

database 

A series of virtual case studies that focus on obtaining informed 
consent and patient motivation relative to managing patients with 
depression. The case studies were published on MedEdPORTAL, a 
peer-reviewed online inventory of educational materials. 

t) Faculty to Student 

Social Media 

Over the course of the semester, a faculty member blogs about 
their experience using wikis for student collaboration and 

Facebook to communicate with students. 
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